Skip to content

Nlp text.rebased#21

Open
zurabksmlp wants to merge 12 commits intomasterfrom
nlp_text.rebased
Open

Nlp text.rebased#21
zurabksmlp wants to merge 12 commits intomasterfrom
nlp_text.rebased

Conversation

@zurabksmlp
Copy link
Collaborator

This pull request is for branch nlp_text.reabsed, it replaces earlier pull request 15 opened on branch nlp_text, called NLP capabillities.

zurabksmlp added 4 commits August 21, 2025 18:22
…pabilites on top of SMLP -- such as ability to accept text as a command and execute it, and also an MCP style invocation of SMLP by an MCP client. These changes are well documented in an updated SMLP manual.

Here are more details:
•	Changes related to moving to tensorflow 2.19.0  and keras 3.8.0, Keras API changed significantly since version keras=3.0.0. Modules affected: train_keras.py, smlp_models.py, and smlp_terms.py.
•	Added initial LLM capabilities: training from scratch, finetuning, and RAG. NLP and text analysis capabilities (text classification, regression, and root cause analysis using subgroup discovery were introduced in previous pool request). Modules: smlp_llm.py, smlp_finetune.py, smlp_rag.py, smlp_generate.py. Also affected: smlp_config.py
•	Added initial agentic capabilities: An SMLP Agent capable to receive command as text and launch SMLP, an SMLP bot and a flow where SMLP Agent interact with another agent. These capabilities are experimental and work in progress. Modules introduced: smlp_agent.py, api_smlp_agent.py, smlp_mcp.py, smlp_chatbot.py.
•	Added initial MCP capabilities to SMLP: created modules smlp_mcp_client.py and smlp_mcp_server.py. The latter is supposed to execute SMLP invocation request sent to it using MCP clients – and smlp_mcp_client.py is such a client that sends request to SMLP_MCP_server.py (currently client-server communication is only supported with local IO – class StdioTransport).
•	Other modules affected: smlp_config.py, smlp_flows.py, smlp_logs.py, and smlp_plots.py (change here was required for MCP support).
•	Regression scripts smlp_regr.py and smp_regr.csv have been updated too: new modes supported, and new tests added.

Virtual env library versions:
Package                     Version
--------------------------- -----------
absl-py                     2.1.0
accelerate                  1.6.0
aiohappyeyeballs            2.4.6
aiohttp                     3.11.13
aiosignal                   1.3.2
altair                      5.5.0
annotated-types             0.7.0
anyio                       4.8.0
asttokens                   3.0.0
astunparse                  1.6.3
attrs                       25.1.0
Authlib                     1.6.1
blinker                     1.9.0
blis                        1.2.0
blobfile                    3.0.0
cachetools                  6.1.0
catalogue                   2.0.10
category_encoders           2.7.0
certifi                     2025.1.31
cffi                        1.17.1
charset-normalizer          3.4.1
click                       8.1.8
cloudpathlib                0.21.0
cloudpickle                 3.1.1
comm                        0.2.2
confection                  0.1.5
contourpy                   1.3.1
cryptography                45.0.6
cycler                      0.12.1
cyclopts                    3.22.5
cymem                       2.0.11
Cython                      3.0.12
dash                        2.18.2
dash-core-components        2.0.0
dash-html-components        2.0.0
dash-table                  5.0.0
dataclasses-json            0.6.7
datasets                    3.5.1
debugpy                     1.8.12
decorator                   5.1.1
deprecation                 2.1.0
dill                        0.3.8
distro                      1.9.0
diversipy                   0.9
dnspython                   2.7.0
docstring_parser            0.17.0
docutils                    0.22
doepy                       0.0.1
eland                       9.0.1
elastic-transport           8.17.1
elasticsearch               8.13.0
email_validator             2.2.0
en_core_web_lg              3.8.0
en_core_web_sm              3.8.0
exceptiongroup              1.3.0
executing                   2.2.0
faiss-cpu                   1.10.0
fastapi                     0.115.13
fastjsonschema              2.21.1
fastmcp                     2.11.3
fasttext                    0.9.3
filelock                    3.18.0
Flask                       3.0.3
flatbuffers                 25.2.10
fonttools                   4.56.0
frozenlist                  1.5.0
fsspec                      2025.3.0
gast                        0.6.0
gensim                      4.3.3
gitdb                       4.0.12
GitPython                   3.1.44
google-pasta                0.2.0
greenlet                    3.1.1
grpcio                      1.70.0
h11                         0.14.0
h5py                        3.13.0
httpcore                    1.0.7
httpx                       0.28.1
httpx-sse                   0.4.0
huggingface-hub             0.30.2
idna                        3.10
imbalanced-learn            0.13.0
importlib_metadata          8.6.1
intel_extension_for_pytorch 2.7.0
ipykernel                   6.29.5
ipython                     8.32.0
ipywidgets                  8.1.5
isodate                     0.7.2
itsdangerous                2.2.0
jedi                        0.19.2
jenkspy                     0.4.1
Jinja2                      3.1.5
jiter                       0.9.0
joblib                      1.3.2
jsonpatch                   1.33
jsonpointer                 3.0.0
jsonschema                  4.23.0
jsonschema-path             0.3.4
jsonschema-specifications   2024.10.1
jupyter_client              8.6.3
jupyter_core                5.7.2
jupyterlab_widgets          3.0.13
kaleido                     0.2.1
keras                       3.8.0
keras-tuner                 1.4.7
kiwisolver                  1.4.8
kt-legacy                   1.0.5
langchain                   0.3.27
langchain-community         0.3.18
langchain-core              0.3.72
langchain-ollama            0.2.3
langchain-text-splitters    0.3.9
langcodes                   3.5.0
langgraph                   0.5.4
langgraph-checkpoint        2.1.1
langgraph-prebuilt          0.5.2
langgraph-sdk               0.1.74
langsmith                   0.3.45
language_data               1.3.0
lazy-object-proxy           1.11.0
libclang                    18.1.1
lightgbm                    4.6.0
llvmlite                    0.44.0
lxml                        5.4.0
marisa-trie                 1.2.1
Markdown                    3.7
markdown-it-py              3.0.0
MarkupSafe                  3.0.2
marshmallow                 3.26.1
matplotlib                  3.7.5
matplotlib-inline           0.1.7
mcp                         1.12.4
mdurl                       0.1.2
meson                       1.7.0
ml_dtypes                   0.5.1
mljar-scikit-plot           0.3.12
more-itertools              10.7.0
mpmath                      1.3.0
mrmr-selection              0.2.8
multidict                   6.1.0
multiprocess                0.70.16
murmurhash                  1.0.12
mypy-extensions             1.0.0
namex                       0.0.8
narwhals                    1.44.0
nbformat                    5.10.4
nest-asyncio                1.6.0
networkx                    3.4.2
nltk                        3.9.1
numba                       0.61.0
numpy                       1.26.4
nvidia-cublas-cu12          12.6.4.1
nvidia-cuda-cupti-cu12      12.6.80
nvidia-cuda-nvrtc-cu12      12.6.77
nvidia-cuda-runtime-cu12    12.6.77
nvidia-cudnn-cu12           9.5.1.17
nvidia-cufft-cu12           11.3.0.4
nvidia-cufile-cu12          1.11.1.6
nvidia-curand-cu12          10.3.7.77
nvidia-cusolver-cu12        11.7.1.2
nvidia-cusparse-cu12        12.5.4.2
nvidia-cusparselt-cu12      0.6.3
nvidia-nccl-cu12            2.26.2
nvidia-nvjitlink-cu12       12.6.85
nvidia-nvtx-cu12            12.6.77
ollama                      0.4.7
openai                      1.78.0
openapi-core                0.19.5
openapi-pydantic            0.5.1
openapi-schema-validator    0.6.3
openapi-spec-validator      0.7.2
opt_einsum                  3.4.0
optree                      0.14.0
orjson                      3.10.15
ormsgpack                   1.10.0
outcome                     1.3.0.post0
packaging                   24.2
pandas                      2.2.3
parse                       1.20.2
parso                       0.8.4
pathable                    0.4.4
patsy                       1.0.1
peft                        0.15.2
pexpect                     4.9.0
pillow                      11.1.0
pip                         25.2
platformdirs                4.3.6
plotly                      5.24.1
plotly-resampler            0.10.0
pmdarima                    2.0.4
polars                      1.22.0
preshed                     3.0.9
prompt_toolkit              3.0.50
propcache                   0.3.0
protobuf                    5.29.3
psutil                      7.0.0
ptyprocess                  0.7.0
pure_eval                   0.2.3
pyarrow                     20.0.0
pybind11                    2.13.6
pycaret                     3.4.0
pycparser                   2.22
pycryptodomex               3.22.0
pydantic                    2.11.7
pydantic_core               2.33.2
pydantic-settings           2.8.1
pydeck                      0.9.1
pyDOE                       0.3.8
Pygments                    2.19.1
PyMuPDF                     1.25.3
pymupdf4llm                 0.0.17
pyod                        2.0.3
pyparsing                   3.2.1
pyperclip                   1.9.0
pysubgroup                  0.8.0
python-dateutil             2.9.0.post0
python-dotenv               1.1.1
python-multipart            0.0.20
pytz                        2025.1
PyYAML                      6.0.2
pyzmq                       26.2.1
referencing                 0.36.2
regex                       2024.11.6
requests                    2.32.3
requests-toolbelt           1.0.0
retrying                    1.3.4
rfc3339-validator           0.1.4
rich                        13.9.4
rich-rst                    1.3.1
rpds-py                     0.22.3
safetensors                 0.5.3
schemdraw                   0.15
scikit-base                 0.7.8
scikit-learn                1.4.2
scikit-plot                 0.3.7
scipy                       1.11.4
seaborn                     0.13.2
sentence-transformers       4.1.0
setuptools                  80.9.0
shellingham                 1.5.4
six                         1.17.0
sklearn-compat              0.1.3
sktime                      0.36.0
smart-open                  7.1.0
smmap                       5.0.2
sniffio                     1.3.1
sortedcontainers            2.4.0
spacy                       3.8.4
spacy-legacy                3.0.12
spacy-loggers               1.0.5
spacy-lookups-data          1.0.5
SQLAlchemy                  2.0.38
srsly                       2.5.1
sse-starlette               3.0.2
stack-data                  0.6.3
starlette                   0.46.2
statsmodels                 0.14.4
streamlit                   1.46.0
sympy                       1.14.0
tbats                       1.1.3
tenacity                    9.0.0
tensorboard                 2.19.0
tensorboard-data-server     0.7.2
tensorflow                  2.19.0
termcolor                   2.5.0
tf_keras                    2.19.0
thinc                       8.3.4
threadpoolctl               3.5.0
tiktoken                    0.9.0
tokenizers                  0.21.1
toml                        0.10.2
torch                       2.7.0
tornado                     6.4.2
tqdm                        4.67.1
traitlets                   5.14.3
transformers                4.52.4
trio                        0.24.0
triton                      3.3.0
trl                         0.17.0
tsdownsample                0.1.4.1
typer                       0.15.2
typing_extensions           4.12.2
typing-inspect              0.9.0
typing-inspection           0.4.1
tzdata                      2025.1
urllib3                     2.3.0
uvicorn                     0.34.3
wasabi                      1.1.3
watchdog                    6.0.0
wcwidth                     0.2.13
weasel                      0.4.1
Werkzeug                    3.0.6
wheel                       0.45.1
widgetsnbextension          4.0.13
wrapt                       1.17.2
wurlitzer                   3.1.1
xxhash                      3.5.0
yarl                        1.18.3
yellowbrick                 1.5
zipp                        3.21.0
zstandard                   0.23.0
Copy link
Collaborator

@fbrausse fbrausse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All in all, seems good. Additional comments: There are plenty of new tests comparing the keras version to 3.0.0 using <=, i.e., with 3.0.0 included. This looks strange to me: why would old API be present in v3.0.0 and below, but not in 3.0.1 or later?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants